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Abstract 

This paper is concerned with cost optimization of an insurance company. The sur- 
plus of the insurance company is modeled by a controlled regime-switching diffusion, 
where the regime-switching mechanism provides the fluctuations of the random envi- 
ronment. The goal is to find an optimal control that minimizes t he total cost up to a 
stoch astic exit time. A weaker sufficient condition than that of ([Fleming and Soner . 
2006, Section V.2) for the continuity of the value function is obtained. Further, the 
value function is shown to be a viscosity solution of a Hamilton-Jacobi-Bellman equa- 
tion. 

Keywords. Regime-switching diffusion, continuity of the value function, exit time 
control, viscosity solution. 
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1 Introduction 

Recently the optimal risk control and dividend distribution problems have drawn growing at - 



tention from researchers. Some recent develoDments can be found in Cadenil 


as et al. 


(20061): 


Chou 


li et al. 


(200; 


3); Preens and Paulsen (2005); Paulsen and Giessind (1997 


): Paulsen et al. 


(2005 


); 

( 


Schmidli 


2001. 2002h: Taksar and HunderurJ (2007): Taksar and Markussen 


(2003): 


Touzi 


2000) 


and the references therein. In those works, the liquid assets of the insur- 



ance companies are modeled by some stochastic processes (usually linear diffusions or jump 
diffusions). At any time t, the insurance companies can choose different business activi- 
ties such as reinsurance, investment, dividend payment, etc. The decisions are based upon 
the information available to them by time t as well as the pre-given economic or politi- 
cal criteria. Different business activities lead to different dynamics in the evolution of the 
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surplus and hence different economic or political returns. This sets a scene for an opti- 
mal stochastic control model for the surplus of the insurance company. In the literature, 
the mos t commonly used cr it eria a r e: (i) maxim izing expected utility at or up to the time 
of ruin ( Irgens and Paulsen J 2005 ): Touzi f 200oh . (ii) m i nimiz ing the probability of ulti- 



mate ruin (Schmidli ( 2001 . 20021 ) ; Taksar and Markussen ( 2003)), a n d (hi) maximiz i ng th e 



cumulative expected discou nted dividends ( Cadenillas et al.l ( 2006 ); IChoulli et al.l ( 20031 ); 



Paulsen and Gjessingl (Il9971 )). 



In contrast to the aforementioned references, in this work, we aim to investigate the 
problem of cost optimization for an insurance company in a regime-switching environment 
using stochastic analysis and stochastic control theories. As we shall see shortly, this is a 
nonlinear optimal control problems in the setting of regime-switching diffusion. In addition 
to the usual operating cost of an insurance company such as corporate debt, bond liability, 
loan amortization, etc, in practice, almost every insurance claim is accompanied by a certain 
amount of business cost resulting from claim appraisal, investigation, settlement negotiation, 
and so on. Minimizing the cost may increase the profit of the insurance company and lower 
premiums for its customers. 

We should also note that the word cost can be used in a general sense: it may represent 
a ny monetary amo unt such as claims, penalties, dividends, utilities, and so on. As illustrated 



m iCai et al.l (120091 ). the total discounted cost actually covers a number of ruin-related quan- 
tities frequently analyzed in ruin literature such as the expected present value of penalty 
at ruin and the total dividends under various dividend strategies. Therefore this work can 
be applied to a broad range of aspects of risk management such as utility and cumulative 
dividends maximization. 

Another feature of this work is the consideration of regime-switching. Most of the exist- 
ing literature on opti mal contro l of ris k processes are based on the framework of diffusion 
approximation model (jGrandelll (ll99lf )). That is, the surplus of an insurance company is 
usually modeled or approximated by a (jump) linear diffusion process. Roughly speaking, if 
the surplus of the insurance company is much larger than the individual claims, then the clas- 
sical h^mogeneous_F\)isson model can be approximated by a diffusion model. Along another 
line, lAsmussenl ( 119891 ) proposed a Markovian-modulated risk model. The model is a hybrid 
system, in which continuous dynamics are intertwined with discrete events. More specifi- 
cally, the evolutions of the surplus (continuous dynamics) is subject to jumps or switches of 
the economic or political environment (discrete events), and the dynamics of the s urplus in 
different environmen ts are markedly different. As demonstrated in lAsmussenl (119891 ) (see also 



Yang and Yinl ( 120041 ) ). this model can capture the features of insurance policies that depend 



on the economic or political environment changes. The states of the discrete event process 
can model for example, certain types of epidemics in health insurance, weather types in 
automobile insurance, the El Nino/La Nina phenomenon in property insurance, or economic 
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conditions in unemployment insurance. Also, in many practical situations, people are more 
concerned with the short term results of the business activities. For instance, the manager 
and/or shareholders of an insurance company want to determine the short-term benefits of 
a particular business activity. Inspired by these arguments, we consider a controlled sur- 
plus process modeled by a regime-switching diffusion (also called a hybrid diffusion in the 
literature) over a finite time horizon [s,T], where T > is a fixed constant and s G [0, T). 

By the nature of the risk process, the insurance company may default at some finite 
time; at which point we say that the surplus is ruined and denote the ruin time by f. 
Consequently, we need only to consider the total cost up to the ruin time or the terminal 
time T, whichever comes first. That is, our control problem is over the interval [s, TAf], with 
TAf = min {T, f } being a random time (stopping time). Thi s is generally termed as exi t time 
control or stochastic control with exit time in the literature ([Fleming and Sonerl ( 120061 )). As 
we shall see in Example 13. 1[ the stopping time TAf depends on the initial surplus X(s) = x. 
As a result, the value function defined in f 12 . 5 j) is not necessarily continuous with respect to x. 
See Example 13.11 for detailed discussions. Then, a problem of great intere st is: Under what 
condition(s), is the value function continuous? (IFleming and Sonerl . 120061 . p. 202) proposed 
a condition on the drift of the underlying (1-dimensional) controlled di ffusion under which 
the co ntinuity of the value funct ion is guaranteed. Th e condition in IFleming and Soner 
(120061 ) was recently generalized in iBayraktar et al.l ( 120101 ) by considering both the drift and 
the diffusion coefficients. In this work, under the more general setting of controlled regime- 
switching diffusion, we propose a new condition in terms of the regularity of the boundary 
point to obt ain the continuity of the va lue function. Our condition is another generalization 
of the one in IFleming and Sonerl (120061 ). See Theorem 13 A\ Proposition 13.71 and Example 13.81 
for more details. 

We emphasize that the continuity of the value function is very important and useful for 
the following reasons. Fi rstly, with the conti n uity o f the value function, one has the dynamic 
programming principle (IFleming and Sonerl (120061 )). which, in turn, helps to establish the 
viscosity solution property for the value function. Secondly, the continuity of the value 
function plays a vital role in the study of numerical approximation to the value function. 
To illustrate, let A be a controlled continuous-time stochastic process and V denotes the 
associated value function. Typically, one constructs a controlled locally consistent discrete 
sequence {X^,n e N} and find the associated value function V h for the discrete problem, 
where h > is the stepsize of the discretization. If the value function V is not continuous, 
then as h J, 0, V h may not converge to V, even though X h converges to X in some suitable 
sense, where X h is the continuous parameter interpolated process of {A^,n 6 N}. In other 
words, X h approximates A, but V h may still differ significantly from V. See Example 13.11 
and also lKushner and Dupuid (1200 ll ) for more details. In fact, this work is largely motivated 
by this aspect of consideration. 
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The classical approach to stochastic control problem is the verification theorem. Typi- 
cally, this approach requires the value function to be sufficiently smooth. In such a case, it 
can be shown that the value function is a classi cal solution of a nonlin ear partial differential 
equation of Hamilton- Jacobi-Bellman type. See lYong and Zhoul (]1999l ) for details. However, 
the smoothness assumption is rather restrictive. As we mentioned earlier, there are many 
examples where the value function is not necessarily sufficiently smooth. In fact, due to the 
dependence of the terminal time T Af on the initial data X(s) = x, the value function in our 
setup may not be even continuous. Then how can we characterize the value function? In this 
paper, we use the notion of viscosity solution. With the aid of the dynamic programming 



principle, we prove that the value functi on is a viscosity solut i on of the Hamilton- 



Bellman equ ation. Since the seminal work lCrandall et al.l ( 119921 ) ; ICrandall and Liond ( 119831 ); 



acobi- 



Liond (119831 ) and others, the viscosity solution characterization have been exploited in many 



control problems under various settings. But the related results in the content of regime- 
switching diffusion is relatively scarce. Moreover, the proof for our case is not a trivial 
extension of the existing results. The presence of regime-switching adds much difficulty in 
the proof. See Theorem 14.31 for more details. 

At this point, it is worth mentioning that thanks to the capability of delineating the 
inherent randomness of many real-world applications, regime-switching diffusions have been 
used in a wide r ange of areas such as finance, biolog y , insurance, etc. S ee for instance 



Hespanhal (120051 ) : Mao and Yuan! (120061 ) : iMaritonl ( 119901 ): lYin and Zhul ( 120101 ) and references 



therein for many prototypical examples of applications of such stochastic processes. On the 
control aspect, optimal control problem s of regime-switc h ing di ff usions have rece i ved growing 
attent i on lately. To name just a few, 



( hoioh : iTaksar and ZeneJ hood ): Win et al l koO$ \: 



Ait Rami et al. 



(2001); 



Li et al. 



Zhou and Yin 



( 120031 ): ISong et al. 



( 120031 ) among others. 



The rest of the paper is arranged as follows. We formulate the problem in Section |2j 
Section [3] is devoted to the continuity of the value function. We obtain several sufficient con- 
ditions for continuity. In Section HJ we show that the value function is a viscosity solution of 
the HJB equation ( 12. lip . We conclude the paper with a few remarks in Section |5j Appendix 
[A] provides a result on regularity of the boundary point. 

A few words about notations are necessary at this point. We use I a to denote the 
indicator function of a set A. If a,b G K., then a A b := min{a,6}. Throughout the paper, 
K is a generic positive constant whose exact value may differ in different appearances. For 
any x G M and r > 0, B(xo, r) = {x G K. : \x — Xq\ < r}. 

2 Problem Formulation 



Let X(t) denote the surplus of a large insurance company at time t G [s, T], where T > 
is fixed and s G [0, T). As we indicated in Section [fl the surplus process X often displays 
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abrupt changes according to dif ferent econ o mic, p olitical, or natural environments facing the 



insurance company. Following lAsmussenl ( 119891 ). we use a continuous time Markov chain 



a(-) to model the variations of the external environments of the insurance company. For 
simplicity, we assume the Markov chain has a finite state space Ai = {l,...,m} and is 
generated by Q = (%), that is, 

or ( + , a +\ -i u\ ■ (\ s +\ /% At + °( At ) ; Kj^h , oU 

P{a{t + At) = j\a{t) = t,a{s),s < t} = < . (2.1) 

[ 1 + qu At + o(At) , if j = i, 

where > for i, j = 1, . . . , m with j ^ i and Yl^jLi Qij = f° r each i — 1, . . . , m. At any 
time t, we denote by u(t) one of the possible business activities available for the insurance 
company. For instance, u(t) may represent a reinsurance strategy, an investment plan, or a 
dividend payment scheme, etc. 

Suppose X satisfies the following stochastic differential equation with regime switching: 
dX{t) = b{t, X(t), a(t),u(t))dt + a(t, X(t), a(t),u(t))dw(t), (2.2) 
with initial conditions 

X(s) = x > and a(s) = a G M, (2.3) 
where w(-) is a standard Brownian motion independent of the Markov chain a(-). Note that 



the in dependence between w and a is a standard assumption in the literature (IMao and Yuan 



( 120061 )). Denote J- t := a (w(r), a(r) : < r < t} . Without loss of generality, we assume that 
J-"o contains all P-null sets. Suppose throughout the paper that the control policy u is {J-'t}- 
adapted and that for any t, u(t) e U, where U is a compact subset of R. Any control u 
satisfying the above conditions is said to be an admissible control. Let U denote the collection 
of all admissible controls. 

Let f := inf {t > s : X(t) = 0} denote the ruin time and set r := T A f. For a given 
control u the expected total cost is 



J(s, x, a, u(-)) = E 



s,x,a 



l(t, X(t), a(t), u(t))dt + g(r, X(r), a(r)) 



(2.4) 



where / : [0,T] xRx M x U R represents the running cost, g : [0, T] x [0, oo) xM^l 
is the terminal cost, and E s )X)CX denotes the expectation with respect to the probability law 
such that the regime- switching diffusion X(t) in (12. 2p starts with initial condition specified 
in (12.31) . As we mentioned in Section [T], we use the word cost in the general sense throughout 
the paper. Hence we allow the functions / and g to be negative. 

The goal is to find an optimal control u* that minimizes the total cost 
V(s,x,a) = J(s,x,a,u*) = inf J(s,x, a,u), \/(s,x,a) G [0,T) x (0, oo) x M.. (2.5) 
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Note that the terminal and boundary conditions are 

V(T, x, a) = g{T, x, a), V(x, a) G (0, oo) x M, (2.6) 

and 

V(s, 0, a) = g(s, 0, a), V(s, a) G [0, T) x M. (2.7) 
Throughout the paper, we assume 

Assumption Al. For each a G Ai, the functions 6(-,-,a, ■), cr(-, ■, a, ■), Z(-,-,a, •), and 
g(-, a) are uniformly continuous. Moreover, there exist positive constants kq and p such that 
for any t G [0, T], x, y G R, a G A4, and u & U, we have 



|<£>(t, x, a, iz) — <£>(t, a, w)| < k \x — y \ , for ip = b,a, I, and g, 
\b(t, x, a, u) \ + \cr(t, x, a, u)\ < n (l + \x\), 
\l(t, x, a, it) | + \g(t,x,a)\ < k (1 + \x\ p ). 



{21 



It is well known (IMao and Yuan! (120061 )) that under Assumption AH], for any u & U and 
any (s,x,a) G [0,T) x (0, oo) x Ai, there exists a unique solution X to ( 12. 2 j) with initial 
condition (12. 3p under the control m. Moreover, J in (12 ,4p is well-defined. In the sequel, we 
denote such a solution by X s,x,a or X s,x,a ' u if the emphasis on the initial conditions and the 
control is needed. Similarly, a s ' a denotes the Markov chain with initial condition a(s) = a. 

For convenience of later presentations, we introduce the operator £f. For any h(t, ■, a) G 
C 2 , t G [0,T],a e M and u G [/, we define 

9 1 <9 2 m 

CfhCt, x, a) = b(t, x, a, u)—h(t, x, a) H — cr 2 (t, x, a, u)-—rh(t, x, a) + > q a jh(t, x,j). (2.9) 



111 



The following veri fi cation the orem can be est a blishe d using the standard argument as 



Fleming and Sonerl (120061) a nd lYong and Zhou! (Il999l ). together with the generalized Ito 



formula (IMao and Yuan! (120061 )). We shall omit the proof for brevity. 

Theorem 2.1. Suppose there exists a function ip : [s,T] x M + xM4 M + such that 

(i) (p(-, •, a) G C 1,2 /or each a G 

(ii) p satisfies the polynomial growth condition, that is, for some positive constants p and 
K, we have 

\(p(t,x,a)\ < K(l + \x\ p ), for any t G [s,T] and a G M, (2.10) 

(iii) (p satisfies the Hamilton- J acobi- Bellman (HJB) equation 
d 

T-<p(t, x, a) + min[£"<£>(t, x, a) + l(t, x, a, u)\ = 0, 

Ot u€U 

t G {s,T),x > 0,a G M, t 2 - 11 ) 
<p(s, 0, a) = g(s, 0, a), <p(T, x, a) = g(T, x,a), s G [0, T), x > 0, a G Ai. 
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Then ip(s, x, a) < J(s, x, a, «(•)) for any initial condition (s, x, a) G [0, T) x (0, oo) x M. and 
any admissible feedback control «(•). 

Moreover, ifu*(-) is an admissible feedback control such that 
d 

— (p(t, x, a) + Of <f(t, x, a) + l(t, x, a, u*(t, x, a)) 
at 

= -^rf(t, x, a) + min[£"y9(t, x, a) + l(t, x, a, u)] (2.12) 

Ot ueU 

= 0, V(t, x, a) e (s, T) x (0, oo) x M., 

Then (p(s,x,a) = V(s,x, a) for all (s,x,a) G [0,T) x (0, oo) x M. and u*(-) is an optimal 
control. 



3 Continuity 

Theorem 12.11 provides conditions under which a sufficiently smooth function coincides with 
the value function. In particular, it indicates that if ip solves the HJB equation (12. lip , then 
it provides a lower bound for the value function V. In addition, if we can find a feedback 
control «*(•) satisfying f)2.12p . then ip = V and «*(•) is an optimal control. It is natural to 
ask whether the converse is true: "Does the value function V defined in (I2.5P always satisfy 
(12. lip ?" In general, the answer is no, since the value function V is not necessarily smooth 
enough (with respect to the variables s and x). More specifically, in our setup, both the 
stopping time r and the control u may depend on the initial value X(s) = x. Consequently, 
the value function may not be even continuous. To illustrate, we consider the following 
uncontrolled deterministic system, where the val ue function is discontinuous . The example 



is inspired by the tangency problem presented in (IKushner and Dupuisl . l200ll . pp. 277-279). 
Example 3.1. Consider 

'dX(t) = 2(t-l)dt, te[s,2] , 

X(s)=x>0, ' 

where s G [0,2]. The solution of (13.11) is 

X(t) = X s ' x (t) = (t - l) 2 + x - (a - l) 2 , t G [s, 2). 

Let r = inf {t > s : X(t) = 0} A 2 and V(s, x) = t. Note that Assumption AQ]is satisfied for 
this example. Consider the case when s G [0, 1]. Then it is obvious that X(t) first decreases 
to its minimum x — (s — l) 2 then increases to oo as t — >■ oo. As a result, we have 

[s-l) 2 >0, 
{s-l) 2 <0. 

Hence it follows that the value function V(s,x) = r is not continuous on the parabola 
{(s, x) G [0, 1] x (0, oo) : x — (s — l) 2 = 0} . See the demonstration in Figure |3~T1 
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X 




T=l f = 2 

Figure 1: Discontinuous Value Function 



Example 13.11 naturally motivates us to the following question: "Under what conditions 
i s the value function con tinuous?" To answer this question, we follow the treatment in 
([Fleming and Sonerl . 120061 . Chapter V) . We first consider an auxiliary control problem, whose 
value function V £ is continuous. Then we propose conditions under which V £ converges 
uniformly to the original value function V as e 4- 0, from which the continuity of V is 
established. As we shall see i n Theorem 13 .4[ Proposition 13 .7\ and Example 13 .8[ our condition 



2006 



Section V.2). 



is more general than that in ([Fleming and Sonerl . 
Let ip :Wx M ^Mbe a function such that 

[■^(x, a) — ip + (y, a)\ < L \x — y\ , for any x, y G R, a G Ai, 

where L > is a constant. For any e > 0, we define 

1 



T £ (t) := exp 



ip + (X(r), a{r))dr > . 



(3.2) 



(3.3) 



Note that T depends on the processes X, a, and the underlying control u as well. But 
for notational simplicity, we have omitted those dependence. Consider the auxiliary control 
problem 



J £ (s, x, a, «(■)) 
V e (s, x, a) 



E fi 



T e (t)l(t, X(t), a(t), u{t))dt + T £ {T)g{T, X(T), a(T)) 



inf J £ (s, x, a, u(-)). 



(3.4) 
(3.5) 



Lemma 3.2. Assume Assumption AQ] and ( 13. 2ft . For any u EU, denote Xi(t) = X s ' Xi ' a ' u (t) 
and 

1 



q(t)=exp 



f(Xi(r) lQ (r))<ir , t e [s,T], 



where s G [0, T), Xi > ; aeM, and % — 1,2. Then we have 

E sup |Xx(t) - X 2 {t)\ 2 <K\ Xl - x 2 \ 
te[o,T] 



(3.6) 
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and 



\r\(t) - r 2 (t)\ < ~(t - s) sup \Xiir) - X 2 (r)\ 

£ re[s,t] 



(3.7) 



Proof. Note that virtually the same argument as that of ( lYin and Zhul . l201dl . Lemma 2.14) 
yields (13. 6ft . Therefore it remains to prove (13. 7\\ . It is easy to see that \e~ a — e~ b \ < \a — b\ 
for any a, b > 0. Thus it follows from ( 13. 2ft that 



|rf(t)-H(*)l 



exp 





1 


< 






£ 




1 


< 






£ 




L 


< 






£ 



ijj + (Xi(r), a(r))dr > — exp 
[V + (^i(r), a(r)) - tfj + (X 2 (r), a(r))} dr 



tp + (X 2 (r), a(r))dr 



r£[s,t] 



This completes the proof. 



□ 



Theorem 3.3. Under Assumption AH the function V s (s, x, a) is continuous with respect to 
the variables s and x for each a G M.,. 

Proof. We divide the proof into several steps. 

Step 1. For <p = l,g and t G [0, T], with notations as in Lemma [3.2[ it follows from the 
Cauchy-Schwartz inequality and Assumption ATTJthat 

E \Tl(t)(f>(t, X 1 (t), a(t),u(t)) - r £ 2 (t)0( S , X 2 (s) , a(s) , u(t))\ 

<E|(r5(*)-r|(*))#,Xi(*),a(t), u (0)| 

+E |r|(t) [c/>(t, X 1 (t), a(t),u(t)) - (f>(t, X 2 (t), a(t),u(t))} \ 

< E 1 / 2 \Tl(t) - T 2 (t)\ 2 E 1 / 2 |0(t, X 1 (t), a(t),u(t))\ 2 

+E 1 / 2 |P 2 (t)| 2 E 1 / 2 |0(t, X 1 (t), a(t), u(t)) - 0(t, X 2 (t), a(t),u(t))\ 2 

< KE X ' 2 \Tl(t) - T £ 2 (t)\ 2 E 1 / 2 (l + |Xx(t)| 2p ) + KE 1 ' 2 \X x {t) - X 2 (t)\ 2 . 

Theorem 3.24), we have 



Note that by virtue of ( IMao and Yuan . 
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E sup |X!(t)| 2p < K = K(x u T,p). 

te[o,T] 



(3.8) 



This, together with Lemma [3 .2\ leads to 

E \Fl(t)cf>(t, X l (t), ait), u(t)) - T%(t)4>{s, X 2 (s), a{s),u(t))\ 



<K-{t-s)E 1 / 2 ( sup iX^r) - X 2 {r)\ ) + KE 1 / 2 \Xi(t) — X 2 (t)\'' 

£ \re[3,t] 

< if |xi — x 2 \ , 



(3.9) 



where K = K(e,xi,T,L,p) does not depend on x 2 , t, or u. 
Step 2. Now it follows from (13. 9 j) that 

|l^ £ (s,xi,a) - l /£ (s,x 2 ,a)| 



< supE 



[rf (t)/(t, a(t),u{t))dt - T £ 2 (t)l(t, X 2 {t), a(t),u(t))\dt 

+ \r 1 (t) 9 (T,x 1 (T))-rmg(T,x 2 (T))\ 



< K \x\ — x 2 \ ■ 



(3.10) 



This shows that V s (s, ■, a) is continuous for any s G [0, T] and a G M. 

Step 3. Next we prove that V s is c ontinuous with respect to s as well. To this purpose, 
we consider s\ < s 2 < T. By virtue of (jFleming and Sonerl . 1200a Section IV. 7), V s satisfies 
the dynamic programming principle. Therefore for any 5 > 0, we can choose a u± G U such 
that 



VHs^x.a) < E 



S2 



r(t)l(t, X{t), a(t),ui(t))dt + V £ {s 2 , X{s 2 ), a{s 2 )) 



< V £ (s 1 ,x,a) + 5/3, 

where T £ (t) = exp{ — ^ f s ip + {X{r), a(r))dr} and X = X Sl ' x ' a ' Ul . Then we have from As- 
sumption AQ]that 



\V £ (s 1 ,x, a) — V £ (s 2 , x,a) \ — 5/3 



< 



E 



»2 



T £ (t)l(t, X(t), a(t), Ul (t))dt + V £ (s 2 , X(s 2 ), a(s 2 )) 

< j 2 K(l + E\X(t)\ p )dt + E\V £ (s 2 ,X(s 2 ),a)-V £ (s 2} x } a 

+ E \V £ (s 2 , X(s 2 ), a(s 2 )) - V £ (s 2 , X(s 2 ), a) | 
:=I + II + III. 

Using (13.81) . we obtain 

' 1 K{1 + E \X(t)\ p )dt < K(s 2 - s x ). 



- V £ (s 2 ,x,a) 



(3.11) 



(3.12) 



si 



For the term last term, we first notice that the definition of V s in (13.51) . Assumption A[U 
and (I3.8P imply that that V £ (s 2 , X(s 2 ), j) < K for every j G Ai, where K = K(x,T,p) is a 
constant. Then it follows that 



III = E \V%s 2 , X{s 2 ), a{s 2 )) - V £ (s 2 , X{s 2 ), a)\ 

= E [\V £ (s 2 , X(s 2 ),a(s 2 )) - V £ (s 2 ,X(s 2 ),a)\I {a{s2 ^ a} ] 

<KP {a(s 2 )^a} 

<K(s 2 - Sl ), 



(3.13) 
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where in the last inequality, we used ( 12. ip . Further, since 

X(s 2 )=x + b(t,X(t),a(t),u 1 (t))dt + a(t,X(t),a(t),Ui(t))dw(t), 

J S\ J Si 

using Assumption AQ]and (13.81) . we can readily verify that 

E\X(s 2 )-x\ < K\s 2 - Sl \ 1/2 . 
Hence by virtue of (I3.10p . we have 

II = E\V £ (s 2 ,X(s 2 ),a)-V £ (s 2 ,x,a)\ <KE\X(s 2 )-x\ < K \s 2 - Sl \ 1/2 . (3.14) 
Combing the above estimates ( I3.12l) -( l3.14l) into ( 13. lip , we arrive at 

\V £ ( Sl , x, a) - V £ (s 2 , x,a)\- 5/3 <K\s 2 ~ Sl | 1/2 + K(s 2 - 8l ). 
Since 5 > is arbitrary, the continuity of V s with respect to s is established, as desired. □ 
Assumption A2. Suppose there exists a function if) : M x Ai i— y R satisfying ( 13.21) and that 

1>(v,j)<0, V(y,j)e[0,oa)xM. (3.15) 
Moreover, there exists some u G U such that 

/ ip + (X(r),a(r))dr > 0, a.s. for any t G (s,T\, (3.16) 

J s 

where X = X s '°' a ' u is the controlled process under the constant control u(t) = u,t £ [s,T], 
a = a s ' a , s e [0,T), and a G M. 

Theorem 3.4. In addition to Assumptions AQ] and ASl suppose also that g(-,-,a) G C 1,2 
for each a G M. and that 

d 

— g(t, x, a) + C^g(t, x, a) + l(t, x, a, u) > 0, V(i, x, a, u) G (0, T) x (0, oo) x M x U, 

(3.17) 

Then V(-, -, a) is continuous for each a G A4. 

Proof We divide the proof into two steps. The first step is concerned with the special case 
when I > and g = while the second step deals with the general case. 

Step 1. First assume I > and g = 0. Fix (s, a) G [0, T) x A4. Let it, A, and a as in 
Assumption AT5J Then (I3.16P implies that for any t > s 



f 1 /"* 
lim T e (t) = lim exp < — / 

eiO eiO { £ Js 



ip + (X(r), a(r))dr > = 0, a.s. 
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and hence by virtue of the dominated convergence theorem and the definition of J £ in (13. 4p . 
we obtain 

lim J £ (s, 0, a, u) = 0. 

e4.o 

Since J £ (s,0,a,u) > V £ (s,0,a) > 0, it follows that 

UmV £ {s,0,a) = 0. 

Let < ei < 62- Then, noting the nonnegativity of the functions ip + and I, we can readily ver- 
ify that J £l (s, 0, a, u) < J £2 (s, 0, a, u) and hence V £l (s, 0, a) < V £2 (s, 0, a). We have shown 
in Theorem 13 . 31 that V s is continuous. Thus Dini's theorem implies that lim £ ^ V £ {s, 0, a) = 
uniformly on [0, T] for each aGM. Now let 

fc(e) := sup 0, a) : s G [0, T], a G M} . 

Then we have lim e ^o h(e) = thanks to the uniform convergence of V s . 

For any (s,x,a) G [0,T] x (0, oo) x Ai, thanks to the dynamic programming principle 
for V s and the definition of h, we have^ 



V £ (s, x, a) 



inf E 



l(t, X(t), a(t),u(t))dt + V £ (t, X(t), a(r)) 

< inf E / l(t, X(t), a(t),u(t))dt + h(e) 

« eW Js 
= V(s, x, a) + h(e). 

Thanks to the definition of J £ in ( 13. 4ft and the assumption that g = 0, we have 
J £ (s, x, a, u) = E / T £ (t)l(t, X(t), a(t),u)dt 

J s 

e\ [ T £ (t)l(t,X(t),a(t),u(t))dt + I {T<T} [ T £ (t)l(t,X(t),a(t),u(t))dt 



Note that for all t G [s, r], G [0,oo). Thus it follows from Assumption A|2] and the 

definition of T £ in (I3.3P that T £ (t) = 1 and hence 



I{r< T} / T £ (t)l(t,X(t),a(t),u(t))dt 



J £ (s, x, a, u) = J(s, x, a, u) + E 

Furthermore, since I > 0, we have 

V(s, x, a) < ^ e (s, x, a) < V(s, x, a) + 



^ote that if r = T, then V £ (T, X(T), a(T)) = V(T,X(T),a(T)) = g{T, X(T),a{T)) = by our 
assumption on g. 
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This implies that V s — > V uniformly on [0,T] x (0, oo) x Ai. Since V s is continuous, V is 
also continuous. 

Step 2. For general / and g, let l(t, x, a, u) = l(t, x, a, u) + J^t, a) + a;, a) and 

g = 0. Then / > by virtue of (13.171) and hence Step 1 implies that the function 

V(s, x, a) : = inf J(s, x, a, u) = inf E / l(t, X(t), a(t),u(t))dt 

is continuous. Apply Ito's formula to g, 

Eg(r, X(t), a(r)) - g(s, x, a) = E jf (j| + £ t u J ^(t, X(t), a(t))dt. 

Then it follows that 



J(s, x,a,u) = E y /(t, a(t),u(t))dt 



= EJ l{t,X(t),a{t),u(t))dt + E^ f^ + cAg(t,X{t),a(t))dt 

= E J l(t, X(t), a(t),u(t))dt + Eg(r, X(r), a(r)) - g(s, x, a) 
= J(s, x, a, u) — g(s, x, a). 

Therefore we conclude that V(s, x, a) = V(s, x, a) + g(s, x, a) is also continuous. □ 

Remark 3.5. With the continuity of the value function at ou r hands, we have the dynamic 
programming principle by virtue of Fleming and Sonerl ( 120061 ): 

V(s,x,a)= inf e( / l(t, X(t), a(t), u(t))dt + V(9 A r, X{9 A r), a{9 A t))\, (3.18) 
«(-)ew [Js J 

where 6 is a stopping time. 

Remark 3.6. Note that Assumption A[2J in particular (I3.16p . is the crucial condition in the 
proof of Theorem 13.41 One may wonder when Assumption AT5] is true? Next we present 
several sufficient conditions. 

Proposition 3.7. Any one of the following conditions implies Assumption AT51' 

(i) There exists a function ip : R x Ai i— > ffi. satisfying (13.21) . (I3.15p . and i/j(0,a) = 
for each a G Ai, and that for some u G U, {ip(X(t),a(t)),t G [s, T]} is a strict local 
submartingale, where X = X s,0 ' a ' ,u , a = a s,a , s G [0,T), and a G Ai. 

(ii) There exists a twice continuously differentiable function ip : M. x Ai h- > M. satisfying 
(13. 2p . (I3.15p . and ip(0, a) = /or eac/i a £ Ai, and that for some u G U , 

£?ip(0, a) = b(t, 0, a, u)ip'(0, a) + ia 2 (t, 0, a, u)ip"(0, a) > 0, (3.19) 
where t G [0, T] and a E Ai. 
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(iii) There exists a u 6 [/ such that the boundary point is regular for the domain (0, oo) 



for the process (X, a) = (X 



s,0,a;u (y s ! a 



), where s G [0, T), a E M. 



Proof, (i) Note that ip(X(s), a(s)) = V>(0, a) = 0. Since {^{X(t), a{t)), t G [s,T]} is a strict 
local submartingale, we have E[?/>(X(t), a(t))|J-y] > i[)(X(s),a(s)) = a.s. for any t G (s,T], 
from which (13 . 16[) follows. 

(ii) This is obvious since ( 13 . 19[) implies that 

^ m 

£?ip(x, i) = b(t, x, i, u)ip'(x, i) + -^{t, ^> h u)if)"(x, i) + ^ Qiji'( x ^ j) > °> * e [ > T ] 

j'=i 

for all G NxAi, where iV is a neighborhood of 0. As a result, {ip(X(t), a(t)),t G [s,T]} 
is a strict local submartingale. 

(iii) If is a regular boundary point, then the function ip(x,a) '■— —x satisfies the con- 
ditions in Assumption AEl We refer to the appendix and Theorem IA.1I for more discussions 
on regular boundary point. □ 

Example 3.8. Consider a uncontrolled surplus process given by 

dX(t) = 2(t - l)dt + (t- X(t)) + dw(t), t G [s, 2], 

with initial surplus 

X(s) = x > 0, 



where s > and w is a one-dimensional standard Brownian motion. Similar to Example 13.11 
we define r = t s ' x = inf {t > s : X(t) = 0} A 2 and V(s, x) = r. 

As in iFleming and Sonerl (120061 ). the signed distance to the boundary point is p(x) = 
—x, x G R. Then 

4«0) = -2( ( -!){>» 

[< for t G (1,2). 

Hence the suffici ent condition (£^(0) > , for all t G [0,2]) for continu i ty of the value 
function given in IFleming and Sonerl (120061 ) fails. See (IFleming and Sonerl . 12006k equation 
(2.8), p. 202) for more details. 

Nevertheless, we claim that is a regular boundary point for the domain (0, oo) and 
hence Assumption AT5] still holds true by virtue of Proposition 13.71 Consequently the value 
function is continuous. To this end, we consider the function (p(x) = —x 2 + x, x G §)■ 
It is easy to see that ip satisfies conditions (i) and (ii) in Theorem IA.11 Next we show that 
ip satisfies condition (iii) in Theorem I A. II as well. In fact, 

/>(o) = 2{t - i V(o) + i ((« - o) + ) V(o) 



2(t -l)-t 2 

-{t- l) 2 - 1 < -1 < 0, Vt G [0,2]. 
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Then it follows that Ct<p(x) < for all t G [0,2] and x G N, where iV C (— |, §) is 
a neighborhood of 0. Consequently, {ip(X(t)),t G [s, 2]} is superharmonic in (0, oo) fl £/. 
Therefore Theorem IA.1I implies that is a regular boundary point and hence Assumption 
AT5] is verified. Further, we can readily verify that all other conditions in Theorem 13.41 are 
satisfied and hence the value function V is continuous. 

Example 3.9. Suppose a controlled surplus process X satisfies 

dX(t) = b(t, X(t), a(t),u(t))dt + <r(t, X(t), a(t),u(t))dw(t), t>s>0, (3.20) 

with initial conditions 

X(s) = x > 0, a(s) = a G {1,2} , 

where if is a one-dimensional standard Brownian motion, a G {1,2} is a continuous time 

f-3 3 \ 

Markov chain generated by Q = I . . , u(t) G [0, 1] denotes the retention rate (so 



1 — u(t) is the proportion reinsured to a reinsurance company) at time t, and 

b(t, x, 1, u) = sini + x+ 0.4 — 0.8(1 — u), a(t, x, 1, u) = sini + 0.5x + 0.5u, 
b(t, x, 2, it) = cost + 3x + 1 — 2(1 — u), a(t, x, 2, u) = cost + x + 2u. 

Note that (13.201) represents a surplus process subject to non-cheap reinsurance, invest- 
ment in a Markovian-modulated Black-Scholes model, and s easonal fluctuations in p r emium 
collection. This is motivated by the model considered in iTaksar and Markussenl (120031 ). 
Denote 

r := inf {t > s : X{t) = 0} A 100. 
The payoff for a reinsurance strategy u(-) is 

J(s, x, a, u) = E Srr>Q / e~ rt X(t)dt, s G [0, 100), x > 0, a = 1, 2, 

J s 

where r > is the discounting factor. The objective is to maximize the payoff and find a 
reinsurance strategy «*(•) such that 

V(s, x, i) = sup J(s, x, i, u) = J(s, x, i, u*), s G [0, 100), x > 0, i = 1,2. (3-21) 

We claim that the value function V is continuous with respect to the variables s and x 
by virtue of Theorem 13.41 In fact, it is obvious that all conditions in Assumptions AQ] are 
satisfied. Next we use Theorem IA.1I and Proposition 13.71 to show that Assumption AT5] is 
also true and hence the claim follows. To this end, we consider (p(x, 1) = —x 2 + 0.5a; and 
<f(x, 2) = —x 2 + 2x, where x G U := (—0.25, 0.25). Then we can easily verify that conditions 
(i) and (ii) in Theorem I A . 1 1 are satisfied. To verify condition (iii), we let u = 0.5 and compute 

£? =0 - 5 (p{0, 1) = b(t, 0, 1, 0.5)^(0, 1) + 0.5a 2 (t, 0, 1, 0.5)y?"(0, 1) - 3y?(0, 1) + 3^(0, 2) 
= 0.5 sint + 0.5(sint + 0.5 ■ 0.5) 2 (-2) 
= - sin 2 1 - 0.0625 < 0, for any t G [0, 100], 
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and 



£y =o - 5 ip(0, 2) = b{t, 0, 2, 0.5)y/(0, 2) + 0.5a 2 {t, 0, 2, 0.5)y/'(0, 2) + 4^(0, 1) - 4ap(0, 2) 
= 2 cost + 0.5(cost + l) 2 (-2) 
= -1 - cos 2 1 < 0, for any t G [0, 100]. 

Hence it follows that ip is superharmonic in ((0, oo) D U) x {1,2} and condition (iii) in 
Theorem IA.1I is verified. Thus by virtue of Theorem IA.1I and Proposition 13.71 we conclude 
that Assumption AT5] holds and hence V defined in (13.211) is continuous. 



4 Viscosity Solution 



With the continuity of the value function and the dynamic programming principle, we can 
now characterize the value function to be a viscosi ty solution of the HJB eq uation (I2.1ip . 
First we recall the notion of viscosity solution from lFleming and Sonerl ( 120061 ) . 



Definition 4.1. A function v is called a viscosity subsolution {viscosity supersolution, resp.) 
of (12.111) if for any •, a) G C 1,2 , aGM, whenever v — (p attains a maximum (minimum, 
resp.) at (s,x,a) with v(s,x,a) = (p(s,x,a), we have 



d 

—ip(s, x, a) + inf {C^(p(s, x, a) + l(s, x, a, u)} > (< 0, resp.) 
at ugu 



(4.1) 



Further, a function v is called a viscosity solution of (12. lip if it is both a viscosity subsolution 
and supersolution of (12. lip . 



We first state a lemma, whose proof can be found in iBayraktar et al.l ( 120101 ). 
Lemma 4.2. For any (s, x, a) G [0, T) x (0, oo) x M. and u <EU, define 

9 := inf {t > s : X{t) £ B(x, h)} A (s + h 2 ), 
where h G (0, 1) and X = X s,x,a ' u . Then there exists a positive constant k such that 

E[0 -s}> nh 2 . 

Moreover, k = k(s,x) is independent of the control u. 

Theorem 4.3. Let Assumptions AH] and AT2] be satisfied. Then the value function ( 12. 5 j) is a 
viscosity solution of ( 12. lip . 



Proof. The proof is inspired by IBayraktar et al.l ( 120101 ) : we use similar ideas. We first estab- 
lish the viscosity subsolution property of the value function V in Step 1, followed by viscosity 
supersolution in Step 2. 
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Step 1. We first prove that V is a viscosity subsolution of (12. lip . Suppose it was not 
the case, then there would exist some if G C 1,2 , a u G U, and a maximizer (so,xo,«o) G 
[0, T) x (0, oo) x M. of V — ip with V(s , x , a ) = (f(s , x , a ), but 

d 



: (p(s , x , a ) + C u so (p(s , x , a ) + l(s , x , a , u) < -5 < 0, 



(4.2) 



dt 

where 5 > 0. Then by the continuity of the function /(•, •, a , u) + (J^ + £")</?(•, •, a ), there 
exists an h G (0, 1) such that 



d 

— ip(s, x, a ) + £%ip(s, x, a ) + l(s, x, a , u) < -5/2 < 0, 
at 



(4.3) 



for all (s,x) G [s ,s + h 2 ) x B(x ,h). Without loss of generality, we assume that h < 1 is 
sufficiently small so that [s , s + h 2 ) x B(x ,h) C [0,T) x (0,oo). Denote X = X s °' x °' a °' u 
and define 

9 := inf {t > s : X(t) £ 5(x , h)} A (s + /i 2 ). 
Note that 6 <t a.s. By virtue of the dynamic programming principle (13. 18 j) . we have 



V(so,x ,a ) < E 



l(r, X(r), a(r), u)dr + V(8, X(0), a(0)) 



L" 7 so 



(4.4) 



Using the assumptions on (p, we can derive from ( I4.4|) that 



< E 



i(r, X(r), a(r),u)dr + <p(9, X(0), a(d)) - <p(s , x , a ) 



*'() 



Apply Ito's formula to <p, 

Eip(6, X(9), ot{6)) - <p(s , x , « ) = E J° (JL + C^j <p{r, X(r), a(r))dr. 
Hence it follows from ( 14. 3 p and Lemma [4.21 that 

d 



< E 



E 



Z(r, X(r), a(r), w) 



dt 



+ C u r \tp(r,X(r),a(r)) 



dr 







l(r, X(r), a ,u) +[-+£?) <p(r, X(r), a ) 



dr + A 



< E / ( — )dr + A 

J s 2 

< --nh 2 + A, 
~ 2 

where k is the constant in Lemma [4.21 and 



(4.5) 



A = E 



*0 



5 



Z(r, X(r), a(r), u) + ( - + C u r ) p(r, X(r), a(r)) 



- Z(r,X(r),a ,u) 



5_ 



£Jf Wx(r),a ) 



dr. 
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Next we show that A is negligible compared to the term —~Kh. To this end, we denote 

H(s, x,a,u) :— l(s, x, a, u) + I — + £" I (p(s,x,a). 

Then for each a E Ai and u G U, as a function of (s, sc), if is continuous and hence bounded 
on the compact [s , So + 1] x -B(x , 1). Therefore we compute from (12. ip that 

A = E f [H(r,X(r),a(r),u)-H(r,X(r),a ,u)]dr 

J s 
f 

<K ^ P {q(r) = j|a(g ) = a } dr 

ps +h 2 

<K (r - s )dr = Kh\ 

J S 

where K is some positive constant independent of h and u. Then it follows from ( I4.5p and 
(14. 6 p that for h > sufficiently small, we have 

< --Kh 2 + Kh 4 < 0. 

This is a contradiction. Hence the value function V must be a viscosity subsolution of (I2.1ip . 

Step 2. Now we show that V is a viscosity supersolution of (12. lip . Again, we use a 
contradiction argument. Suppose on the contrary that V was not a viscosity supersolution 
of (12.111) . Then there would exist a <fi G C 1 ' 2 and a minimizer (s , x , a ) G [0, T) x (0, oo) 
oiV — (f> with V(sq, x , «o) — 4>( s o, ^0; a o) ; but 

d 

—(p(s ,x ,a ) + inf 0(a o ,ac o ,ao) + i(so^o,«o,«)} = 5 > 0, (4.7) 

where 5 is a constant. By Assumption A(U the function 

<9 

(s, x) i — ^ —(f){s, x, a) + inf {£"0(s, x, a) + Z(s, x, a, «)} 

is continuous for each q G M. Hence we can find an h > such that 
d & 

— (fi(s,x,a )+ inf {£>(s, x, q ) + l(s,x,a ,u)} > -, W(s,x) G [s , s +/i 2 ) xB(x , h). (4.8) 
or wee/ 2 

Let £ = |k/i 2 , where k is the constant in Lemma T4.2I Let u G IA be an e-optimal control and 
denote X = X s °> x °> a °' u . Put := inf {t > s : X(t) £ B(ar , ^)} A (s + h 2 ). Then it follows 
that 

V(s ,x ,a )>E f l(r,X(r),a(r),u(r))dr + EV{6,X(0),a{6))-e. (4.9) 

</s 



As in Step 1, using the assumptions and Ito's formula on 0, we can rewrite (14. 9p as 



> E 
= E 



l(r, X(r), a(r), u{r))dr + <f)(0, X(0), a(6)) - 4>(s , x , a ) 



So 



— e 



s L 



l(r,X(r),a(r),u(r))+[ ^ + ) </>{r, X(r), a(r)) 



dr — e. 



But e = jKh 2 . This, together with Lemma 14. 2 [ leads to 



> E 



E 



l(r,X(r),a(r),u(r))+[^ + C< r) ) <f>(r,X(r),a(r)) 



dr - 6 -E[d - s ] 



l(r, X(r), a{r),u{r)) 



| + ^" (r) ) 0(r,X(r),a(r))- J 



E 



l(r, X(r), ao, «(r)) + ( ^ + C (r) ) ^(r, X(r), a Q ) - | 



dr 
dr + B, 



(4.10) 



where 



B = E 



so 
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Z(r, X(r), ot(r), u(r)) 4 ( ^ + £^ (r) ) 0(r, X(r), a(r)) 
i(r, X(r), a , «(r)) - + C u r ^ } 0(r, X(r), a ) 



dr. 



Using the same argument as that in Step 1, we deduce that for some constant K independent 
of h and u, 

\B\ < Kh 4 . (4.11) 



Therefore it follows from f l4~8]) - ffl~TT|) that 

> E / (- - -)dr - \B\ > -Kh 2 - Kh 4 > 0, 
" J S0 K 2 4 J 1 '"4 

for h > sufficiently small. This is a contradiction. Therefore V is a viscosity supersolution 
of ( 12. lip . This completes the proof. □ 

5 Conclusions and Remarks 



In this work, we considered cost optimization problem for an insurance company. The 
surplus of the insurance company was modeled by a controlled regime-switching diffusion. 
We presented a sufficient condition for the continuity of the value function and further 
characterized it as a viscosity solution of the HJB equation ( 12.111) . Compared with the 
usual diffusion models, the consideration of regime-switching mechanism provides a better 
approximation to the real-world dynamics. The novelty of this work also includes a new 
sufficient condition for continuity of the yalue function. Th e suffi cient condition in this 
paper is a new generalization of the one in lFleming and Sonerl (120061 ). 
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A number of other questions deserve further inv estigations. In particu lar, the next log- 
ical step is to establish a strong comparison result ( jCrandall et al.l ( 119921 )). which, in turn, 
implies that the value function is the unique viscosity solution of the HJB equation (12. lip . 
Consequently, we have the complete characterization of the value function. It is conceiv- 
able that due to the presence of regime switching, the analysis will be much more involved 
than that in the user's guide ICrandall et al.l (119921 ) . Also, we were not able to obtain the 
explicit form of the value function and an optimal control by solving (12. lip . The reason for 
this is that (12. lip is a coupled system of nonlinear second order partial differential equa- 
tions, rendering extreme difficulty in finding a closed form solution of (12. lip . Therefore a 
viable alternative is to employ numerical approxi mations. The Markov chain approximation 
method developed in lKushner and Dupuid ( 120011 ) will be utilized in the near future. We may 
also consider more complicated stochastic models such as regime- switching diffusions with 
jumps as well as other optimality criteria such as dividend maximization and ruin probability 
minimization problems. 



A Regular Boundary Point 

This appendix provides a result on regular boundary points. For notational simplicity, 
we shall present the result when the continuous component X is 1-dimensional. The multi- 
dimensional case can be handled in a similar fashion. Note that the notations in the appendix 
are not necessarily the same as those in the main part of the paper. 

Let (X, a) G R x Ai be a switching diffusion process, where Ai = {l,...,m}. The 
generator Q of [X, a) is defined as follows. For any h(-,-,i) G C 1 ' 2 , i G Ai, we define 

d 

Qh(t, x, i) =—h(t, x, i) + hit, x, i)b(t, x, i) 
at 

x rn (A.l) 

+ -h"(t,x,i)a 2 (t,x,i) + S ^q ij h(t,x, j), 

where h' and h" denote the first and second order derivatives of h with respect to the variable 
x, respectively, b, a : [0, oo) xlxjM i->- R are given functions, and qij are constants satisfying 
qij > for i ^ j and qu = — X^/i Qij- Further, we assume that b and a satisfy 

\b(t,x,i) -b(t,y,i)\ + \a(t,x,i) - a(t,y,i)\< K\x-y\, 

(A.2) 

\b(t,x,i)\ + \a(t,x,i)\ < K(l + \x\), Vt G [0, oo), x,y E R,i G M, 



where K is a positive constant. It is well-known ( Mao and Yuan! ( 120061 ) ) that under these 
conditions, for any s > 0, x G R, and a G Ai, the generator (lA.ip uniquely determines a 
switching process (X(-),a(-)) with initial conditions X(s) = x and a(s) = a. Denote such a 
process by (X s,x,a , a s,a ) if the emphasis on initial conditions are needed. 
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Let G be an open subset of K and a E dG. The point a is said to be regular for the 
process (X, a) in G if for any s > and aeMwe have 



P{r = s} = l, 



where 



r = r s ' a ' Q := inf {t > s : X s ' a ' Q (t) £ G} 
denotes the first exit time for the process (X, a) from G. 

Theorem A.l. The point a E dG is a regular point if there exist a neighborhood U of a and 
a function if : U x Ai i— y R swc/i i/iat 

(i) y?(x, 2) > /or all x E G HU — {a} and each i E M; 

(ii) lim^a^gG <f(x, i) = for each i E M. ; and 

(iii) ip is superharmonic in {G fl U) x JM, that is, if is bounded below and continuous in 
(G C\U) x M and satisfies 

<f(x, a) > Ev(X s ' x ' a (r v ), a s > a (r v )), V(x, a) E V, (A.3) 

where s > 0, V C (G n U) x M, and t v = inf {t > s : (X 8 ^'"^), ^ 1/}. 

Chapter 13), we use similar ideas. We 



1965 



Proof. The proof is motivated by (iDynkin . 
divide the proof into several steps. 

Step 1. Let U and p be as in the statement of the theorem. Without loss of generality, 
we assume 

sup f(x,i) < 1. (A.4) 
(x,i)eUxM 

In fact, it is not hard to see that if p satisfies (i)-(iii), then so does the function <p A 1 = 
min(<y9, 1). 

Suppose that a is not regular, then we have 



P {t > $o} > for some s > and i E M, 



(A.5) 



where r = r s °' a,e = inf {t > Sn : X so,a ^ (t) ^ G}. Then, by virtue of the Blunmenthal Zero- 
One Law ( iKaratzas and Shrevd ( 1l99ll )). 



Obviously, 



P{r>s } = l. 



{X(sq) = a,a(so) = £, r > s } C sup dist(a, X(t)) > 

te[s ,r) 



(A.6) 
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Hence it follows that 



P < sup dist(a,X(i)) > } > 0. 

te[s ,r) 



Therefore for some 5 > 0, we have 



P{ sup dist(a,X(t)) > 5 } > 0. 

t£[s ,r) 



(A.7) 



Step 2. Now set G = G n B(a,5) and r = inf {t > s ,X(t) <£ G }. Note that by 
choosing 5 sufficiently small, we may without loss of generality assume that Go C U. Now 
for any t > Sq, we can write 

E [l{T <TMX{T ),a{T ))] 

to<t,to> 

4} ^(X(ro),«(r ))] (A.8) 

:=/!(t) + / 2 (t). 

Using (1A.4I) . (1A.6j) . and the continuity of the sample paths of X, we have 



lim/i(t) < limP {r < t} < P {r = s } = 0. 

t^SQ t^SQ 



(A.9) 



On the other hand, it follows from the strong Markov property and ( 1A.3I) that 

h{t) = E [j {TO<T>TO>t} ^(X(T Q ),a(To))] 

= E [l{T >t}Ex(t)Mt) [I{t <tMX(t ), a(r ))]] (A.10) 
<E[/ {ro> ^(X(t),a(t))]. 

Thanks to condition (ii), for any e > 0, we can choose a neighborhood N of a such that 

y?(x,i) < e, for any (x,i) e (N C\G) x M. (A- 11 ) 



Al so, since the s ample paths of X are continuous (see, for example, iMao and Yuan! (120061 ) 
or 



Yin and Zhul (120101 )). we can choose some D C N such that 

P {p > So } = 1, where (5 = inf {t > s : X(t) £ D} . 



(A.12) 



Then it follows from ([Qjl . (tOOl) . and (lATll that 

/ 2 (t) <E[J {7D>t} ^(Jf(t),a(t))] 

= E [/ {ro>ti/3>t}V 9(X(t), a(*))] + E [l {T0>t ^ t} p(X(t), a{t))] 
<e + P{/3<t}. 



By virtue of (1A.12|) . we further obtain limsup t i So fyt) < e. But e > is arbitrary, it 
therefore follows that 

lim/ 2 (t) = 0. (A. 13) 

tlso 
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A combination of (1A.8j) . f ]A.9j) . and ( 1A. 13|) leads to 

E[/ {ro<T} ^(X(ro),a(r ))] =0. 
Step 3. Now we set A = {r < t,X(t ) G G}. Then it is obvious that 



(A. 14) 



sup dist(a,X(t)) > 5 } C A. 

te[s ,T) 



This, together with ( 1A.7I) . implies that P(^4) > 0. Therefore it follows from condition (i) 
that 

E [I {tq<t}V (X(t ), a(r ))] > E [p{X{r ), a(r ))I A ] > 0. (A.15) 

Finally, the contradiction between (1A.14I) and (1A.15j) implies that a must be a regular bound- 
ary point. □ 



Remark A. 2. If the process (X,a) is assumed to be strong Feller (jZhu and Yinl (120091 )) . 
then we can show that the cond itions in Theorem I A. II are also necessary The argument is 



similar to that in (IDynkin 



1965 



Chapter 13). We shall omit the details here. 
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